DOCUMENT RESUME 



ED 317 622 



TM 014 752 



AUTHOR 
TITLE 



PUB DATE 
NOTE 



PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Eash, Maurice J.; And Others 

Changing Learning Environments in High School 

Science: An Evaluation of the Results of an NSF 

Workshop. 

Apr 90 

16p.? Paper presented at the Annual Meeting of the 
American Educational Research Association (Boston, 
MA, April 16-20, 1990). 
Reports - Research/Technical (143) — 
Speeches/Conference Papers (150) 

MFOl/PCOl Plus Postage. 

Biology? Chemistry? Classroom Techniques? ^Curriculum 
Development? *Educational Environment? Educational 
Improvement? High Schools; Inservice Teacher 
Education; Learning Modules? Program Evaluation? 
»Science Teachers? ^Secondary School Science? 
Seminars? ^Summer Programs? ^Teacher Workshops? 
Teaching Methods 

National Science Foundation? Our Class and Its Work 
Questionnaire 



ABSTRACT 

The effects of a 3-week summer workshop sponsored by 
the National Science Foundation (NSF) and seven seminars held in the 
following academic year on the learning environments of 40 
participating teachers' high school biology and chemistry classes 
were assessed. In this 1988-89 program, the 40 teachers visited 10 
industries and agencies to observe applications of basic science* 
From these observations, they developed, in the university 
laboratories, teaching modules for their classroom curricula. 
Development activities were augmented by lectures, demonstrations, 
and presentations by visiting scientists. Videotapes were prepared to 
demonstrate basic science applications and processes as part of the 
teaching modules. This evaluation focused on the ability of the 
inservice program to stimulate effective teaching approaches to 
theories and concepts in biology and 'Chemistry for secondary sz ool 
science teachers, to build curriculum units from industrial and 
societal applications of conceptual and syntactical science using 
teaching strategies geared toward a range of student abllitien, and 
to provide lesser-prepared teachers an opportunity to build enriched 
curriculum units and develop effective teaching strategies. 
Evaluation data were gathered via a learning environment measure 
called "Our Class and Its Work." Results indicate that the program 
did improve the science education of participants, stimulating 
positive student attitudes and areater student achievement. Five data 
tables are included. (TJH) 
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The purpose of this investigation was to evaluate the effect of a specially designed three 
v.'eek summer workshop and seven seminars held in the following academic year on the learning 
environments of the forty participating teachers' high school Chemistry and Biology classes. * 
In this project the forty participating teachers visited ten industries and agencies to observe 
applications of basic science. From these observations they developed in the University 
laboratories teaching modules for their classroom curricula. The development activities were 
augmented by lectures, demonstrations and visiting scientists. As submitted for funding the 
project listed six objectives: 

(1) to stimulate effective teaching approaches to theories and concepts in Biology and 
Chemistry for secondary science teachers who are teaching in several science 
disciplinary fields 

(2) to build curriculum units from industrial and societal applications of conceptual and 
syntactical science using teaching strategies geared to a range of students' abilities 

(3) while the major purpose of the project is teacher enhancement, a secondary focus is to 
provide training for teacher-leaders who will be identified and encouraged to give 
leadership in their schools and to science education in the Commonwealth 

(4) to provide lesser prepared Biology and Chemistry teachers (especially those teaching in 
other than their major) an opportunity to build enriched curriculum units and develop 
more effective teaching strategies 

(5) to develop long-range netwoiKing opponunities for secondary science teachers in the 
Boston metropolitan area to work together and profit from a continuous exchange of 
new materials and teaching strategies 

(6) to provide impetus and incentive for development of long term collaborative 
relationships between secondary science teachers, college and university scientists, and 
industry and government-affiliated scientists 

A comprehensive evaluation design was constructed to ascenain the accomplishment of these 
six objectives. In this paper the emphasis will be primarily on objectives 1, 2 and 4 which focus 
on directly improving the effectiveness of the teacher in the classroom. Activities of the workshop 

* These workshops, seminars and their evaluation, were funded by a grant from the National 
Science Foundation's Directorate in Science and Engineering Education's Teacher Enhancement 
Program to a project, entitled "Applications of Basic Science in Industry and Society to 
Enhance Secondary School Science." The results and their interpretations are solely the 
responsibility of the authors. 
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were directed to getting teacliers to explore new developments in science and to build teaching 
modules with less reliance on textbooks and more student participation in the construction of the 
learning activities. These teaching modules were then tested in their classrooms during the regular 
academic year and reported back in the monthly seminars for discussion and critique. At the end of 
the academic year they were then edited, bound and distributed to all class participants. A numlier 
of video tapes were prepared demonstrating basic science applications and processes as pan of the 
teaching modules. These were commercially edited, duplicated and distributed to the participants at 
the end of the year. 



METHOD 

Twenty Chemistry and twenty Biology teachers were selected from two pools of 76 
applicants who responded to advertisements for the workshop and seminars. They were chosen lo 
represent a mix of experienced and inexperienced teachers, the latter were defined as having 5 or 
less years of teaching experience. (See Table I) Attention was given to including teachers who 
were in schools that had a larger than average minority enrollment. Participants schools had \5% 
minorities enrolled where state-wide, representation of minorities in the Massachusetts population 
is slightly less than 8%. The teachers were from both suburban and urban schools, and averaged 
15 years of teaching experience. Previous surveys of Commonwealth high school science teachers 
reported two findings that guided the development of the workshop and the selection of 
participants: (1) less than well-prepared teachers were filling high school science positions, 
especially in chemistry, and (2) the aging science teacher population had not been prepared in the 
newly emerging, most significant developments that were shaping the applications of science in 
government agencies and industries. A major working assumption in the design of the workshops 
and seminars was that the learning en^'ironment in the participant teachers' classroom would be 
improved if teachers experienced direct application of basic science in industries and governnieni 
agencies and developed as teams instructional modules which they taught and critiqued with peers 
in a supportive setting. Science teaching in general has suffered from didacticism, teacher-centric 
and textbook dominated teaching (Stake and Easley, 1978). These teaching methods have been 
especially resistant to change and take on an ideological rationale among some science teachers 
(Tobin, 1988). Broadening teacher experience in constructionist approaclies, » e., shaping 
learning activities according to student purpose and disciplinary objectives rather then covering 

2 



ERIC 



4 



material (Tobin and Eraser, 1989), and sharpening participants' critical judgments of teaching in a 
supportive setting are fundamental to breaking the customary teacher-centric dominated classroom. 
An external and internal evaluator were used to assess the project. An external evaluator was u.sed 
to develop an instrument and administer it for the purpose of gathering data on the extent of 
implementation of the design of the workshops and seminars and to obtain panicipants' reactions 
to the workshops and seminars. The external evaluator also processed and analyzed the third 
category of data, a measure of the learning environments in participants' classrooms and in three 
comparison groups. 

Because learning environments in the classroom are so crucial to the development of student 
attitudes toward science and shapes not only their present but future achievement (Talton and 
Simpson, 1986), a learning environment measure. Our Class and Its Work (OCIW) (Hash and 
Waxman, 1983) was used as the principal comparative measure. 

The modified OCIW, contains 9 scales com^^osed of 50 items. As a validated instrument, 
eight of the scaies gather data on student perceptions of teachwr behaviors tnat have been found to 
raise or lower student achievement. (Waxman and Hash, 1983) A 9th scale was added to OCIW 
which gathers data on students' perception of science as a subject, as a potential career, and as a 
contributor to social policy, (for the scales and a sample item of each, see Tab! ^ IV) Th^ OCIW 
was administered: 

(1) as a pre-post measure in participant teachers' classroom 

(2) to another science teachers' class in the same school with similar students 

(3) to students in the classrooms of a second group of teachers who had been accepted for 
the AY89-9() workshop and seminars but had not taken the AY88-89 workshop and 
seminars. 



The results of the three different categories of data are carried in tables II, 111,1V and V. 
Tables II and III present the reactions of teachers to the workshop and seminars and the perceived 
utility and usefulness of the activities in their classrooms. Sixty one % of the teachers said they 
utilized the activities which were targeted by the workshop and designed to promote 
constructionists approaches to teaching. The six activities used by three quiuiers of the teachers 
were: used a different method to teach a concept or major learning (75%), introduced new 
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materials into regular curriculum (93%), shared my project experience and materiuls with other 
teachers (93%), included in classroom work, djrect applications of science concepts in industry 
(82%), focused specific lessons on importance of science in society and specific community (75%) 
and expanded the science content taught into new areas this year (82%). Teacher participants rated 
workshop experiences on a four point scale (1: extremely valuable, 4: not valuable) gave the 
different summer workshop experiences a mean rating of 1.48. They rated as hi §; nest the 
relationships with University scientists (x* 1.13) and other teacher participants (x 1.18) and as 
lowest the relationships with scientists in agencies (x 2.00) and industries (x 1.70). In two of the 
three comparisons on the QCrW the workshop participants had significantly higher results. 

The OCIW which was the chief objective measure of the classroom learning environments 
collected three sets of data. It was assumed in the evaluative model that the workshops and 
seminars impact on teachers' knowledge of and teaching of science would be reflected in 
participants' classroom performances when compared with teacher control groups. On the OCIW 
improved scores of teacher classroom behaviors as perceived by students have been found to 
reflect greater student achievement in the subject, i.e., science. The OCIW result of the 
comparisons pre vs post-test scores of participants, workshop participants vs non-workshop 
participants, 1988 participants vs 1989 participants (pre-test), and experienced teachers vs 
inexperienced teachers, and boys vs giris in workshop participants classes are shown in tables IV 
and V. 

The combined workshops of Biology and Chemistry were significantly higher on 7 of the 9 
scales in the post-test than the pre-test (See Table V). The Biology workshop a!one did not show a 
significant difference on the overall pre vs post-test. On the comparison between panicipants vs 
non-participants' classes there was a significant difference ^n favor of the workshop participants' 
classes. A somewhat surprising finding was the 1989 NSF participants overall pre-test scores 
were significantly higher than the 1988 post-test scores. We believe after nearly a year's 
experience with the 1989-90 worksiiop panicipants that they are, as these aata reflect, a superior 
group of teachers and reflect selective recruitment by the 1988 workshop participants. Experienced 
teachers, as expected, performed significantly better than inexperienced teachers. There was no 
significant difference overall on the OCIW for boys vs gkls, though in the comparisons of the 
individual scales there were two significant differences, one favoring boys.and one favoring giris. 



D IS CUSSION 



Curriculum evaluation has never been a favorite field for investigation by researchers as 
Wayne Welch observed. The difficulty of developing random assignment to treatment(s) or even 
of producing matched studies is legendary. Nevertheless the consistency of outcome measures and 
the repeated replication of findings does provide some confidence in the belief that the changes as 
observed, measured and contained in the self reports are not simply due to chance. 

While the results of these comparisons are not the products of a true experimental design 
(random assignment to treatment), the outcomes based on two different comparisons (pre and post, 
and other science classes) do consistently reflect significant differences that favor the classes of the 
participants who have received the treatment of the summer workshop and follow-up seminars in 
1988. (See Tables IV and V) 

Overall, workshop participants from the 1988 workshop and seminars showed personal 
growth in all nine of the OCr ,V scales. When 1988 panicipants were compared with other science 
teachers in their schools they showed gains in those behaviors which stimulate student subject 
achievement in six of the nine scales. When 1988 participants post tests, i.e., measures taken after 
they had completed the summer workshops, taught their modules, and been through the academic 
year seminars, were compared to their pre-tests, they were significantly higher in seven of the nine 
scales. In the other two scales while not achieving significance, the trends were in favor of the 
1988 workshop participants. 

The 1988 workshop participants when compared on their post-test scores with the 1989 
piirticipants pre-test scores (both having been administered in May 1989) were significantly higher 
on the three scales of Enthusiasm, Structuring Comments and Task Orientation. The 1989 
participants were higher on the three scales on Instructional Time, Opportunity to Learn, and 
Pacing. There was no significant difference on the three scales of Didactic Instruction, Feedback, 
and (Student) Attitude toward Science. These findings confirmed the Pis' belief that the two 
workshops were composed of different teachers. The 1989 participants appear as a group to be 
more motivated and have a greater professional orientation than 1988 participants. This greater 
professionalism has been manifest in their better attendance at the academic year seminars and in 
the greater progress they have made in preparing teaching modules. 

Student observations of teaching behaviors have been found to be highly accurate indicators 
of teachers' classroom functioning and strong predictors of students' level of learning of subject 
matter. After completing the workshop, the 1988 participants were viewed by students as being 
more enthusiastic and using more and different materials in their classrooms, behaviors thai 
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stimulated student interest in science. Students of teachers who finished the workshop were much 
more likely to strongly agree with the statement, "Students are more interested in science", than 
their counterpart control students in classes whose teachers had not been in the workshop. 

Workshop teachers upon completion of the 1988-89 experience were viewed far more often 
by their students to use more effective techniques of instruction that promote students' learning 
through: immediate feedback, providing appropriate challenging assignments, and keeping 
students' task oriented. Teacher performance was improved in using community resources for 
science, and providing greater variety in classroom instruction. Most importantly for the future of 
science the students of workshop participants strongly agreed with the statements, "I understand 
the relationship of science to industry" and "I am interested in a career in science." at a statistically 
significantly higher rate. Boys and girls did not differ in their response to these statements. In 
both the teachers' self-report data in Tables II and III and the students' observation data in Tables 
IV and V there is consistence in these two data sets in describing changed teacher behavior and 
improved teacher performance in the classroom. 

The comparative results on experienced vs inexperienced teachers and boys vs girls 
(Comparisons Table IV) (4 and 5) reflect two other significant outcomes. The designation of 
experienced and inexperienced teachers based on preparation and teaching is supported in the 
empirical data as experienced teachers significantly ranked higher on the OCIW than inexperienced 
teachers on both pre and post measures in two comparisons. However the inexperienced teachers 
made greater gains on the two scales of Pacing and Attitude Toward Science (See Table V). 
Comparison five which examined the treatment effect on boys and girls produced one of the most 
interesting and important findings. Boys and girls did not differ significantly but on two scales. 
Enthusiasm was higher for boys and Opponunity to Learn was statistically significantly higher for 
giris. From these results we conclude their growth in achievement in science does not differ 
greatly. Given that the preponderance of data in previous research on gender differences in .science 
finds boys performing better in both attitude and achievement, this is an unusual, unanticipated 
outcome (Becker, 1989). Moreover, we believe the finding of little difference in boys and giris' 
results in the 1988 workshop participants' classes, if substantiated by future studies, potentially 
underscores an important contribution of this type of workshop to both Biology classes (which are 
mostly required science), and to Chemistry classes (which are generally elective) to attracting more- 
women into science careers. 

De Rose and others (De Rose, 1979) have commented on the critical role of the teacher in 
establishing the quality of the learning environment. The teacher is the key player in ensuring an 
environment which promotes continuing interest in science, but there are some structural 
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impediments which preclude teachers following a more constructionist approach to curriculum — 
the short 40-45 period that most high schools schedule and the lack of good serviceable equipmeni 
to use in teaching and demonstrating. Schools seem to be both short of budget for new equipmeni 
and even to be unable to keep equipment repaired and in operating condition. Teachers' failure to 
bring students into the planning of instruction (see Table II) and the persistence of didactic 
instruction (see Table V) we believe is related to the 40 minute period. 

Of particular note is the increase in interest in science in the students of workshop 
participants. This seems contrary to the national trend of diminished interest in science for upper 
adolescents and young adults (Hofstein et al, 1986, and Yager, Penich, 1986). The need for 
building enthusiasm and interest in science heightens when one looks at the problem of the 
diminishing numbers of students choosing science majors as under-graduates and pursuing 
advanced degrees in science. Recently the President of AAAS produced an analysis which 
demonstrated that students who had taken ten or more semesters of science and mathematics in 
high school but did not enroll in any type of science program constitute a large pool of potential 
science manpower that in itself would erase the projected shortfall in the next two decades, if they 
were induced to enroll in science and engineering programs -- even allowing for subsequent 
anrition (Atkinson 1990). 

Do these NSF workshops and seminars effect longer range changes in students' attitudes 
and achievement in science? The current measures are not adequate to give a definitive answer to 
this crucial question. However, the consistency of these data favoring the NSF workshop 
participants' classes, do point strongly toward the belief that the workshop treatments are directly 
contributing to improving teacher performance that leads to higher student achievement in science 
(Biology and Chemistry). Moreover, these teaching behaviors strongly stimulate interest in 
science. 

What are the variables in the workshop and seminars' design that contribute to the 
effectiveness of the treatment? The evaluative data indicate that the prior assumptions guiding the 
original design of the workshop are supported. Teacher performance affecting student 
achievemCiit is improved by: 

(1) expanding teachers' h^^rizons on new developments and applications of science in 
industry and government 

(2) having teachers generate insn-uctional modules stimulates classroom interest and builds 
teaching co'' petency 

(3) providing a supportive setting for critical examination of try outs of teacher generated 
materials 
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(4) fostering a supportive network of relationships between university scientists and high 
school science teachers 

(5) continuing a program that maintains the network of relationships established by the 
cooperative atmosphere of the NSF workshop 

Thus we conclude from the evaluative data that the 1988-89 workshop and se- anars linked 
with a continuing program of activities did improve the science education efforts of those teachers 
who participated. These efforts in turn stimulate more positive student attitudes towards science 
and greater student achievement. Progress along these two lines advances national goals for 
science as well as promotes the scientific accomplishments of individual boys and girls. 
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TABLE I 



EXPECTED AND /^CTUAL ENROLLMENTS OF 

COHORT GROUPS IN BIOLOGY AND 
CHEMISTRY SUMMER 1988 WORKSHOPS 





Expected (Percentage) 


Actual (Percentage) 


Minorities 


10 


7.5 


Women 


50 


45.0 


Physically-disabled 


2 


0.0 


Well-prepared Teacher 


60 


62.5 


Under-prepared Teachers 


40 


37.5 



Note: The project participants have a heavier representation in students taught who are 

classified disadvantaged and minorities than originally targeted. In the original proposal, 
minorities and disadvantaged students taught by participants were targeted at an expected 
rate of 20 percent and 10 percent respectively. In the teacher participants selected they were 
found to be 25 percent and 15 percent of the school populations. The state- wide 
representation of minorities in Massachusetts population is slightly less than 8 percent. 
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tabu: II 

UTILITY OF WORKSHOP EXPERIENCES 
ACTIVITIES IN CLASSROOM ENHANCEMENT 
(Participants Self-Report) 



Activity or Experience Percen. -ge 

Have talcen a field irip(s) tiiat replicated those in NSF Workshop 42 

Have used new equipment in my teaching this year 64 

Have used a different method to teach a concept or major learning * 75 

Have introduced new materials into regular curriculum * 93 

Have had students plan and organize cooperative group projects* 43 

Have changed my approach to incorporate more student planning 2 1 
in new curriculum 

Have brought more resource people from industry into my classroom 29 

Have Shared my project experience and material with other teachers 93 

Have included in classroom work direct applications of science concepts 82 
in industry * 

Have observed increased student interest in science as a career this year * 54 

Have increased use of questioning in my class this year * 57 

Have found students with a wider range of academic abilities more 50 
interested in science this year * 

Have focused specific lessons on the importance of science in 75 
society and specific community * 

Have expanded the science content taught into new areas this year * 82 



* Item verified by student observation data from a standardized instrument. 
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TABLE 111 



RATING OF THE OVERALL WORKSHOP 
AND EXPERIENCES BY PARTICIPANTS (YEAR I) 



Overall Workshop Percentage 

(Responding positively) 

Overall I found project experience valuable 95 

Project will enhance my teaching 100 

As resul t of the site visits my development of classroom 100 
teaching units will be enhanced or greatly enhanced 

Activities Rating s ( 1 extremely valuable, 4 not valuable) x Rating 

Lectures 1 .40 

Site Visits 1.40 

Experiments 1.50 

Seminars 1.54 

Relationships with Scientists in Industry 1 .70 

Relationships with Scientists in Universities 1.13 

Relationships with Scientists in Agencit? 2.00 

Relationships with Other Participants 1.18 
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TABLE IV 



COMPARISONS OF IMPACT OF WORKSHOPS AND SEMINARS 

ON 1988 PARTICIPANTS' 
CLASSROOM TEACHING BEHAVIORS AND 
STUDENT ACHIEVEMENT (OCIW) 



GROUP COMPARISONS 

1. PRE and POST TESTS: 

NSF Participants' Classes 

Biology 

Chemistry 

Combined Biology & Chemistry 



Level of 
Si gnificance 



469 0.3125 - 0.5681 NS 
1047 0.9352 2.9692 0.003 
1516 0.3717 - 0.7906 NS 



2. 1988 NSF PARTICIPANTS' CLASSES 
COMPARED TO NON NSF CLASSES: 

Combined Biology & Chemistry 1988 

Combined Non NSF Classes 



572 0.4395 2.6912 0.007 
310 



3. 1988 NSF PARTICIPANTS (Post-Test May 1989) 
COMPARED TO 

1989 NSF PARTICIPANTS (Pre-Test May 1989): 
Combined Biology & Chemistry 1988 
Combined Biology & Chemistry 1989 



634 0.9513 - 6.4418 0.001* 
672 



4. EXPERIENCE TEACHERS 1988 
COMPARED TO 
INEXPERIENCE TEACHERS 



442 0.4782 14.85 0.01 



5. BOYS COMPARED TO GIRLS 



625 0.1148 2.12 



NS 



* The 1989 participants significantly outperformed the 1988 patticipanls -- see text. 



14 



TABLE V 

IMPACT OF WORKSHOPS AND SEMINARS 
ON 1988 PARTICIPANTS' CLASSROOM TEACHING BEHAVIORS 

AND STUDENT ACHIEVEMENT 
(OCIW, SCALES COMBINED BIOLOGY AND CHEMISTRY) 



SCALES AND DESCRIPTION 

(Reliabilities in ( ) reported in Chronbach Alphas] 



Level of Significance 
(.05 and above reponed) 



Other 1988 
Pre Science vs 
Post Classes 1989 



Experienced Boys 

vs vs 
Inexperienced Girls 
Teachers '88 1988 



.05 .01* 



.01 .01* 



NS .01* 



NS 



NS 



NS 



.01*** 



.05** NS 
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Didactic Instruction(.87^ 

Implies that the teacher controls and directs the instruction .05 NS NS NS NS 

for all students in the class. 

Sample item: "Our teacher lets us do things our own way." 
Enthusiasm (.92) 

Considers the extent to which a student sees the teacher .01 .01 .01 NS .01 

exhibit excitement and interest in teaching. 
Sample item: 

"We try new and different things in our classroom." 
Feedback (.91) 

Describes the extent to which the teacher respond to NS .001 NS NS NS 

students answers and provide students with feedback 
about their schoolwork. 

Sample item:"Our teacher carefully checks all our work." 
Instructional Time (.86) 

Refers to the time students spend in learning. NS 
Sample item:"We are always working in our class." 
Opportunity to Learn (.92) 

Indicates how well the teacher provides opportunities for .05 
all students to learn or cover criterion material. 
Sample item:"Many students do not finish their work." 
Pacing (,9 2 ) 

Deals with whether or not the classroom work is at the .0 1 
appropriate level of difficulty for students in the class. 
Sample item: 

"Our teacher spends too much time going over work." 
Structuring Comments (.91) 

Refers to whether the teacher provides overviews at the .01 .01 .05 NS NS 

beginning and end of instructional sequences and 
whether students understand. 

Sample item:"Our teacher often reviews yesterdays work." 
Task Orientation (.84) 

Indicates the extent to which the classroom is businesslike. .05 NS .01 NS NS 

Sample item: "We always have an assignment to work on." 
Attitude Tow ard Science (.77) 

Indicates students attitudes toward importance of science .01 .05 NS .05** NS 

in society, the scientific method in their lives, and 

science as a chosen career. 

Sample item: "Scientists improve our lives." 

* The mean differences favored the control group (see text) 

** The inexperienced teachers made greater gains on these two scales than experienced teachers but the 

difference in means favored the experienced teachers. 
*** Boys had higher means than giris on "Enthusiasm scale" and girls had higher means on "Opportunity 
to Learn scale". 
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